Optimizing thermodynamic trajectories using evolutionary and gradient-based reinforcement learning

Using a model heat engine, we show that neural network-based reinforcement learning can identify thermodynamic trajectories of maximal efficiency. We consider both gradient and gradient-free reinforcement learning. We use an evolutionary learning algorithm to evolve a population of neural networks, subject to a directive to maximize the efficiency of a trajectory composed of a set of elementary thermodynamic processes; the resulting networks learn to carry out the maximally-efficient Carnot, Stirling, or Otto cycles. When given an additional irreversible process, this evolutionary scheme learns a previously unknown thermodynamic cycle. Gradient-based reinforcement learning is able to learn the Stirling cycle, whereas an evolutionary approach achieves the optimal Carnot cycle. Our results show how the reinforcement learning strategies developed for game playing can be applied to solve physical problems conditioned upon path-extensive order parameters

Fault healing inferred from time dependent variations in source properties of repeating earthquakes

We analyze two sets of repeating earthquakes on the Calaveras fault to estimate in-situ rates of fault strengthening (healing). Earthquake recurrence intervals t, range from 3 to 803 days. Variations in relative moment and duration are combined to study changes in stress drop, rupture dimension, rupture velocity, and particle velocity as a function of tr. Healing rates and source variations are compared with predictions of laboratory derived friction laws. Two interpretations of event duration τ are used: one in which τ: is given by the ratio of slip to particle velocity and one in which it scales as rupture dimension divided by rupture velocity. Our data indicate that faults strengthen during the interseismic period. We infer that source dimension decreases with tr due to aseismic creep within the region surrounding the repeatinge vents. Stress drop increases 1-3MPa per decade increase in tr, which represents an increase of a factor of 2-3 relative to events with tr between 10 and 100 days. This rate of fault healing is consistent with extrapolations of laboratory measurements of healing rates if fault strength is high, on order of 60MPa, ands tress drop is roughly 10% of this value

Optimizing thermodynamic trajectories using evolutionary and gradient-based reinforcement learning

Using a model heat engine, we show that neural network-based reinforcement learning can identify thermodynamic trajectories of maximal efficiency. We consider both gradient and gradient-free reinforcement learning. We use an evolutionary learning algorithm to evolve a population of neural networks, subject to a directive to maximize the efficiency of a trajectory composed of a set of elementary thermodynamic processes; the resulting networks learn to carry out the maximally-efficient Carnot, Stirling, or Otto cycles. When given an additional irreversible process, this evolutionary scheme learns a previously unknown thermodynamic cycle. Gradient-based reinforcement learning is able to learn the Stirling cycle, whereas an evolutionary approach achieves the optimal Carnot cycle. Our results show how the reinforcement learning strategies developed for game playing can be applied to solve physical problems conditioned upon path-extensive order parameters.Comment: 11 pages, 5 figure

Optimizing thermodynamic trajectories using evolutionary and gradient-based reinforcement learning

Using a model heat engine, we show that neural network-based reinforcement learning can identify thermodynamic trajectories of maximal efficiency. We consider both gradient and gradient-free reinforcement learning. We use an evolutionary learning algorithm to evolve a population of neural networks, subject to a directive to maximize the efficiency of a trajectory composed of a set of elementary thermodynamic processes; the resulting networks learn to carry out the maximally-efficient Carnot, Stirling, or Otto cycles. When given an additional irreversible process, this evolutionary scheme learns a previously unknown thermodynamic cycle. Gradient-based reinforcement learning is able to learn the Stirling cycle, whereas an evolutionary approach achieves the optimal Carnot cycle. Our results show how the reinforcement learning strategies developed for game playing can be applied to solve physical problems conditioned upon path-extensive order parameters

Optimizing thermodynamic trajectories using evolutionary and gradient-based reinforcement learning.

Using a model heat engine, we show that neural-network-based reinforcement learning can identify thermodynamic trajectories of maximal efficiency. We consider both gradient and gradient-free reinforcement learning. We use an evolutionary learning algorithm to evolve a population of neural networks, subject to a directive to maximize the efficiency of a trajectory composed of a set of elementary thermodynamic processes; the resulting networks learn to carry out the maximally efficient Carnot, Stirling, or Otto cycles. When given an additional irreversible process, this evolutionary scheme learns a previously unknown thermodynamic cycle. Gradient-based reinforcement learning is able to learn the Stirling cycle, whereas an evolutionary approach achieves the optimal Carnot cycle. Our results show how the reinforcement learning strategies developed for game playing can be applied to solve physical problems conditioned upon path-extensive order parameters

The effect of loading rate on static friction and the rate of fault healing during the earthquake cycle

The seismic cycle requires that faults strengthen (heal) between earthquakes, and the rate of this healing process plays a key role in determining earthquake stress drop, rupture characteristics and seismic scaling relations. Frictional healing (as evidenced by increasing static friction during quasi-stationary contact between two surfaces is considered the mechanism most likely to be responsible for fault strengthening. Previous studies, however, have shown a large discrepancy between laboratory and seismic (field) estimates of the healing rate; in the laboratory, rock friction changes by only a few per cent per order-of-magnitude change in slip rate, whereas seismic stress drop increases by a factor of 2 to 5 per order- of-magnitude increase in earthquake recurrence interval. But in such comparisons, it is assumed that healing and static friction are independent of loading rate. Here, I summarize laboratory measurements showing that static friction and healing vary with loading rate and time, as expected from friction theory. Applying these results to seismic faulting and accounting for differences in laboratory, seismic and tectonic slip rates, I demonstrate that post-seismic healing is expected to be retarded for a period of several hundred days following an earthquake, in agreement with recent findings from repeating earthquakes